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We investigate a control process described by a linear system of or- 
dinary differential equations with a noise of special type acting to the 
control parameter. As the cost functional the probability of the final 
state vector to enter to a given set in the phase space is considered. 
Necessary conditions of optimality (of the Pontryagin maximum prin- 
ciple form) and existence theorems are developed. The initial control 
problem was trasformed to an auxiliary deterministic problem, the 
differentiability of the auxiliary functional was discussed. 
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Abstract 



1 Introduction 



Consider the optimal control problem 



|x = A (t) x-K (t) B (t) u (t) +f(t), < t < 1, 
x(0) = x , 



(1) 




u(-)eil 



max 



(2) 



where x is the system state vector and u is the control one, Q is a given 
nonrandom goal set in the phase space, £ (t) is a scalar random process of 
the given type. 



The necessary conditions for optimality in the form of Pontryagin max- 
imum principle and existence theorems are obtained in this paper keeping 
on our recent investigations [THS]. In sec. [2] we state the problem, in sec. [3] 
reduce it to the equivalent one of deterministic type. Existence of optimal 
control is proved in sec. HJ Conditions of differentiability of the functional, 
necessary and sufficient conditions for optimality are obtained in sec. 0, El 
respectively. 

2 Statement of the problem 

Denote by (■, ■) the inner product and by |-| the Euclidean norm of a vector 
(matrix) in IR n . Let (fi, P) be a complete probability space. Forming the 
system ([1]) deterministic matrices A (t), B (t), f (t) are of order nxn, nxm, 
n x 1, respectively; they have summable with respect to Lebesque measure 
components; the ones of matrix B (t) are quadratically summable; x is a 
nonrandom vector, Q is a nonrandom Borel's set in M. n . 

Let {£i (a;) , . . . , £tv (w)} be a given set of random values on P), 

{0 = t < ti < . . . < t N = 1} 

a finite subset of [0, 1]. Now we introduce the process £ (t, u): 

= 6(w) , t e i = TJJ. 

The set il of admissible controls consists of all measurable deterministic 
functions u (i) , i 6 [0, 1], taking their values in a fixed set V C M n . For every 
admissible control u (•) G [0, 1] there exist a unique (in the class of process 
with absolutely continuous trajectories) solution of the Cauchy's problem 
(JIJ) [4, p. 35]; the corresponding value of the goal functional <p (u (■)) G [0, 1]. 
Optimal control problem (DQ)-© can be posed as the problem of choosing of 
an admissible control u° (•) G il provided maximal value to the functional 

^( u °(-)) = m a X¥?(u(-)). 
u(-)eii 
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(3) 



3 Auxiliary deterministic problem 

Using Cauchy's formula for the solution of a linear differential equation sys- 
tem one can write 

x n (1) = $ (1) (x + / S- 1 (s) (f (s) B (s) u (s) + f (s)) <h) -- 

N U TV . 

= x + $ (1) £ & / (s) B (s) u (s) ds = x + £ 

7=1 7=1 

where 

x = $(l) jx + J ^- 1 (s)f(s)ds 

z« = $(l) y $- 1 (s)73( S )u(s)rfs, (4) 

$ (i) is a fundamental matrix for the system ([1]) ($ (0) is the unitary matrix). 
Note that 

z« = %y {U) - * f _ iy (^0 , * f = $ (1) (tf) , i = MT, (5) 
where y M (i) is the solution of the Cauchy's problem 

|y =A(t)y+5(t) u, t £ (0, 1) , 
y(0) = 0. 

Taking into account Ej) and (jSJ) and introducing the functions of N vector 
variables 



TV 

g(zx,...,z N ) = P ( £)&Zj G Q 



N 

G(yi,...,y JV )=P( Ex^y.eQ 



(6) 



7=1 



where Q = Q - x , %i = & - i = 1,N -1, Xn = Cn, we get the 
following representation for the functional (j2]): 



P(U(;)) = P(X B (1)6Q) 

x + Erf e QJ 
f } ) =G(yu(«i),...,y« (tor))- 



Pfxo+ErfeQ) = ( 7 ) 



7=1 

(1) 

£ IZ^,...,Z 



It is clear now that the stochastic control problem ([T]) is equivalent to the 
next deterministic control problem: 

f |y =A (t) y+B (t) u, t G (0, 1) , 

\y(o) = o, W 

y? (u (■)) = G (y u (ti) , . . . , y« (tjv)) -> max . 

u(-)eu 

Note that the form of the last is unusual: the cost functional is of terminal 
type and it depends on control process states, taken at several moments of 
the time of control. 



4 Existence theorem 

Let {yi, . . • , yjv} be an arbitrary set of vectors in K n . Construct a new 
vector Y in M. M , M = n ■ N by the following rule: Y = {y 1; . . . ,Yn} = 
{y{, . . . , Ui, yl, ... , y%}- Consider the following functions 

N 

i=i 

h (Y) — P(u:S (u, Y) e Q j . 

Lemma 1. If the set Q is closed in W 1 , then h (•) is upper semicontinues 
function. 

Prove. Let Y* G M M , Y^ — > Y* as k — > oo. Consider random events 

^ = |w : S(u,Y k ) G Q| , 

A* = tu :S(u,Y*) G q|, 
B= limsupAfc = n U Aj. 

k— >oo k>l j>k 

If u G B, then there exist a sequence fc m — > oo such that w G A fcm , in 
other words 

S(w,YJgQ 

for evey m. Proceeding in this statement to the limit as m — > oo and taking 
into account that the function S (u, •) is continuous and the set Q is closed 
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we obtain S (co, Y*) G Q, so w e A*. Hence B C A* and P (5) < P (A). 
On the other hand, due to measure properties [5J p. 34] 

P (B) = P (limsup A fc ) > limsup P (A k ) . 

\ fc— >oo / fc— ^oo 

Finally we have 

h (Y*) = P (A*) > P (5) > limsup P (A fe ) = 
= limsup /i (Yjfe) . 

This proves the lemma. 

Theorem 10. Suppose that ii is a weakly compact in L™ [0, 1] and £/ie 
goal set Q is closed in R n . Then the optimal problem has a solution. 

Prove. Let {u^ (•)} be a maximizing, weakly converging to u° (•) G il 
sequence of controls: 

ipiui (•)) -> sup^(u(-)) < 1. 

It is easy to prove that y U; — > y u o (tk), k = 1, N. According to lemma 1 
we have 

p(u°(0) = G(y u o (ti),...,y u o (tjv)) > 
> lim sup G (y U; (ti) , . . . , y U; (t N )) = 

= lim^(uj (■)) = sup (f(u(-)) . 
'-*- 00 u(-)eU 

This proves the theorem. 

Another approaches to the existence of the solutions of such kind problems 
were developed in p~H3]. 



5 Conditions of differentiability 

Differentiability of the functional with respect to the phase coordinates plays 
important role in necessary and sufficient conditions of optimality. Is this 
section we study the differentiability of functions g, G. 

Restrict our consideration to the case of absolutely continuous with re- 
spect to Lebesque measure random vector £ = . . . ,£at}. Let / be the 

1 This theorem was proved by A.Yu. Zorin 
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probability density function of £, Z = {z^, . . . , z^} be a given set of vectors 
in R n , 

O (Z) = {r eR N : nz* + . . . + r N z* N G Q} . 

If N > n then the set O (Z) is unbounded in M. N even Q is bounded in M. n . 
Let iTj be the projection of O (Z) to the hyperplane rj = 0. It is easily shown 
that for any convex Q the line passing through an arbitrary point pj G 7ij 

in parallel to the axe rj intersects O (Z) over interval rj G ( r j 1 \ r j 2 ^j the 

functions r,- = rj (pj) are bounded for a bounded Q and z* 7^ 0. 
For a fixed k £ 1,N consider the function 

h k (z fc ) = g (zj, . . . , z£, z fc , z£ +1 , . . . , zy , z fe G R n . 



Lemma 2. Let j,k G l,iV, vector z* 7^ 0, £/ie probability density 
f (r 1? . . . ,r N ) is differentiable with respect to rj and the function $kj ( r ) = 
d{rkf) /drj is summable in ~R N . Then the function hk(-) is differentiable 
at the point z* k in the direction of vector z* . The corresponding directional 
derivative 

^ = ±h k (z* + ez*) | e=0 = - J (r) dr. (9) 

J O(Z) 

Prove. We have 

/i fc (z* + £ z*) = P {£iz* + . . . + £ fe z£ + . . . + G Q} = y /„ (r) dr, 

(10) 



O(Z) 

where is the distribution density of the vector 



7/ 



*7< = 6, «^ j, 

Vj = tj + ^k, i = j, 



r f \ — p / \ _ f / ( r i> • • • ' r i-i> r i - £r k, ■ ■ ■ , r N ) , fc, 



We see that 



dF J -^^r (ri, . . . , rj- - er fc , . . . , r^) , j 7^ fc, 

6 



Consider the function 



O(Z) 



O e (Z) 



where £ (Z) = P e O (Z), P e is a linear transformation operator in M. N : P £ = 
Ibiill) P»i = 5jj - a (e) t^fc, 

f a (e) = e, k ^ j, 

1 a(e) =e/(l + e), fc = j, 

5jj is the Kronecker delta. We have 



*(e)|< J |^-(r)|dr, 



so the function \l> (e) is integrable in a neighborhood of the point e — 0. Let 
us prove that the function is continuous at this point. Let denote for R > 

K fi = {r : |r| > R} , 1^ ee R w \ir fi) 

o £ , R (z) ee o £ (z) n k r , o' £>R (z) ee o £ (z) n ^, 
o 0)R (z) ee o (Z) n (z) ee o (z) n ^. 

As the function (r) is summable in then for every 5 > there exist 
i? > such that 



/ 

E 



\&kj(r)\dT <- 



for every measurable subset E C K R . For _R > do the estimation 



|*(e) 



J <&kj (r) dr- J § kj (r) dr 



< 



§ kj (r) dr 



3e,fl(Z) 



3e(Z) 



®kj (r) dr 



O(Z) 



< 



3o,h(Z) 



(r) dr- y (r) dr 



The sum of the first two terms is less than 25/3. As the Lebesque measure of 
the symmetric difference between bounded sets 0' e R (Z) and O' R (Z) tends 
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to zero as e — > 0, then the last term is less than 5/3 for all sufficiently small 
e. So we have \ty (e) — ^ (0)| < S for all such e. This proves the continuity 
of \I> (e) at the point e = 0. 

Listed above is sufficient for the possibility of differentiation of the integral 
(HOD at the point e = 0|p. 132]: 

Sr/f < o -> *=-/*««*■ 

O(Z) O(Z) 

This proves the lemma. 

For convex Q we have from fl9]) 

«9z* " J {TkJ) 'r y =rf> ( Pi ) ^ 

where in the case of infinite values of ' (p^) the substitution should be 
settled zero. Note that using this representation of the derivative one can 
prove lemma 2 for more wide assumptions about / do not presuming the 
existence of the partial derivatives. 

Theorem 2. Suppose that N > n, k e 1, N, set of vectors {z{, . . . , z*} C 
Z forms a basis in R n , the density f is differentiable with respect to rj, and 
functions (r) = d (rjt/) /drj are summable in R N . Then the function 
hk (zfc) is differentiable at the point z£ and its gradient is 




where {e 1 , . . . , e n } is a dual basis for {z*, . . . , z*}. 

Prove. If we prove the differentiability of the function, then after de- 
composing of the gradient with basis {e*} and each of vectors e 5 with basis 
{z*} [7, p. 229] we receive 

i=l j=l 3 i=l 

that proves formula ([TT]) . 

To prove the differentiability of the function h k it is sufficiently to check 
the continuity of the derivatives |^ (z^) at the point z£. After decomposition 
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of an increment Az^ with the basis {z*}, Az^. = Aiz* + . . . + A n z*, Aj = 
(Azfc, e 1 ) we obtain by analogy with ( ITU]) 

gfr (zj + Az fc ) = - J i- (r k f) dv = 

3 O(Z) 

= rfc / $,y(r)dr=*(A fc ), 

O a (Z) 

where 

/a (r) = ^/ (ri - i^r fc , . . . , r n - ^r*, r n+1 , ...,!>), 

A (Z) = P A (Z) , Pa^MI, 

= I ~ i^ fc > * = li^ 

Pl[ I z = n+l,iV ' 

We can to prove that ^ (A&) — >■ ^ (0) as Az^. — > similar to lemma 2; 
this proves the continuity of the derivatives and the theorem too. 

It is clear that in the statement of the theorem one can change {z*} to 
any other basis Z in MJ 1 . If dimension of the linear span £ of Z is less than 
n, then under formulated in the theorem properties of / we can guarantee 
only differentiability of over subspace £. 

It is easy to construct the examples of functions h k (z^) which are not 
different iable or even not continuous at the given point z£ (see below). The 
reason of these may be in absence of such properties of Q as bodility, con- 
vexity, boundary smoothness. 

Example 1. 

n = 3, N = 2, Q = {xx = 0, (x 2 - 2) 2 + (x 3 - 2) 2 < 1} , 
z* = (0,0,1), z* = (0,1,0). 

Example 2. 

n = 2, N = 1, Q = { Xl > 0, x 2 > 0, 
1 < x\ + x\ < 2}, z* = (0,1). 

Example 3. 

n = 2, N = 1, Q = {(xi - 7) 2 + X 2 . < 25, 
(x 2 -7) 2 + x 2 <25}\ z* = (l,l). 
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Theorem 3. Let function f be smooth in R , Q be a ball in R n . For 
every set of vectors Z, which does not belong to hyperplane holding the ori- 
gin and touching the boundary of Q ; the functions hk(z k ), k = 1,N are 
differentiate at points z£ G Z. 

Using the analytical form of Q, we can prove theorem 3 by direct calcu- 
lations. Note that for any set of vectors belonging to tangential plane the 
functional equals zero; it can be maximal only for a degeneration problem 
(see sec. [6]). Hence the statements of the theorem are natural. 

6 Conditions of optimality 

Denote by Z (u (■)) = <^z^- ) , i = 1, iV j the set of vectors in M. n , described by 

formulas (j4]) for a given control u(-) G il . The problem ([1]) is called a) 
degenerate if the functional <p (■) is constant in il ; b) regular for the control 
u* (•) G il , if for any k — 1, N the functions hk(zk), defined for the set 
Z (u* (■)), are differentiable at the corresponding points z *) (see sec. [5]). 

Lemma 3. If x G Q, then the sufficient condition for u° (•) G U to 
be optimal is that the set Z (u° (■)) be trivial (i.e. consists of zero vectors 
only). On the contrary, if the problem is not degenerate and Xq G" Q, then 
the necessary condition for u° (•) G U to be optimal is that the set Z (u° (•)) 
be nontrivial. 

Prove. If z^o = 0, i = 1,N, then from ((7|) we have tp (u° (•)) = 
P (x G Q). The corresponding value of the probability equals 1 = sup <p (u (•)) 
for x G Q and = inf if (u (•)) < sup ip (u (•)) for x G" Q. This proves both 
statements of lemma. 

Theorem 4. Let u° (•) be optimal control, and the problem (T7J) is regular 
for the control u° (■). Then for almost all t G [0, 1] 

(9 (t) , B (t) u° (t)) = max (9 (t) , B (t) v) , (12) 
where the conjugate function 

N 

0{t) = zZ* k ^ 9k (*)> 

k=l 

£e k = -A' (t) 9 k 

9 k (1) = Vh k (z2 } ) , 
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and Xk (t) is the indicator of the set [tk~i, t k ) , k = 1, N. 
Prove. Let r G (tk-i,t k ), < e < t k — r 



v G V, te(r,T + e), 
u°(t), t#(r,T + e). 



Taking into account that for such kind of variation only vector z q varies, 
we have 

<p (u £ (•)) - V? (u° (•)) = (zS } ) - ^ (z«) = 
= (v/* (z5 ) ),A £ z( fc ))+o(|A £ zW|), 

A £ z« ee z« - Z W = 
= $ (1) / (s) B (s) (v - u° (s)) ds. 

r 

Using the Lebesque theorem about differentiation of integrals we obtain for 
almost all r G (tk-i,tk) 

8<p = Vm{<p{vf{-))-<p{ii {-)))/e = 
= (Vh k (zj?) , $ (1) (r) B (r) (v - u° (r))) = 
= (6>(r) I S(r)(v-u°(r))>. 

Because the control u° (■) is optimal, the variation Sep < 0; this implies (|12|) . 
The theorem is proved. 
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